Non-verbal Vision-based Interfaces

All computer input devices serve one essential purpose. They transduce some motion or energy from a human agent into machine useable signals. One may therefore think of input devices as the `perceptual organs' by which computers sense the intents of their human users. Since vision is the primary sense by which humans receive interaction from others, we expect computer vision to play a concomitant role in computer interaction. We outline this role, highlight the challenges to the development of vision-based interfaces, and propose an approach for overcoming these impediments. Prospective vision research areas for HCI include human face recognition, facial expression interpretation, lip reading, head orientation detection, eye gaze tracking, three-dimensional nger pointing, hand tracking, hand gesture interpretation, and body pose tracking. For vision-based interfaces to make any impact, we will have to embark on an expansive approach which begins with the study of the interaction modality we seek to implement. We illustrate our approach by discussing our work on vision-based hand gesture interfaces.